Combining Forecasts using Clustering

Given sales forecasts for a set of items along with the standard deviation associated with each forecast, we propose a new method of combining forecasts using the concepts of clustering. Clusters of items are identified based on similarity in their sales forecasts and then a common forecast (or combined forecast) is computed for each cluster of items. The objective of clustering is to minimize the mean square error (MSE), which is the sum of the variance and squared bias of the combined forecasts. It is easy to show that combining forecasts from a group of items generally decreases the variance but increases the bias of the combined forecast. A new clustering method is proposed based on this tradeoff between the decreased variance and increased bias. A useful property of the proposed clustering method is that it automatically finds the right number of clusters. On a real dataset from a national retail chain we have found that the proposed method of combining forecasts produces significantly better sales forecasts than either the individual forecasts (forecasts without combining) or an alternate method of using a single combined forecast for all items in a product line sold by this retailer.